Chapter 2 



A Review on Differentiation 



Reading: Spivak pp. 15-34, or Rudin 211-220 



2.1 Differentiation 

Recall from 18.01 that 

Definition 2.1.1. A function f : R n -> R m is differentiable at a G R n if 

there exists a linear transformation A : R n — > R m snc/i toa£ 

Mm l/(q + ft)-/M-A(h)| =0 (21) 

/i->o \h\ 

The norm in Equation 2.1 is essential since /(a + h) — f(a) — X(h) is in R m 
and /i is in R™. 

Theorem 2.1.2. // / : R n -> R m differentiable at a G R n ; toen ^ere zs 
a unique linear transformation A : R n — > R m i/ia£ satisfies Equation (2.1). 
H^e denote A to &e Df(a) and call it the derivative o// a£ a 

Proo/. Let fj, : R n -> R m such that 

Um l/( B + fc)-/M-^)| =0 (22) 

ft^o \h\ 
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and d(h) = f(a + h) — f(a), then 

Um \\(h)-»(h)\ = Um \\(h) - d(h) + d(h) - »(h)\ 

h^O \h\ \h\ 

h^O \h\ h^Q \h\ 

= 0. (2.5) 

Now let h = tx where t G R and x G R n , then as t — > 0, trr — > 0. Thus, for 
a; 7^ 0, we have 

| A (to) -//(ta;) | = |A(ar) - /x(x)| g 

= (2.7) 

Thus = X(x). □ 

Although we proved in Theorem 2.1.2 that if Df(a) exists, then it is 
unique. However, we still have not discovered a way to find it. All we can do 
at this moment is just by guessing, which will be illustrated in Example 1. 

Example 1. Let g : R 2 ^ R be a function defined by 

g{x,y) = In x (2.8) 

Proposition 2.1.3. Dg(a,b) = X where A satisfies 

\(x,y) = --x (2.9) 
a 

Proof. 

\g(a + h,b + k)- g(a,b)-X(h,k)\ | m(a + h) - m(a) - ± • h\ 
hm = hm 

(h,k)-*0 \(h,k)\ (h,k)->0 \(h,k)\ 

(2.10) 
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Since In' (a) = ^, we have 



ln(a + h) - ln(a) - - ■ h\ 
lim^ ; ,, , V ! a - ^=0 2.11 

h^o \h\ 



Since \(h, k)\ > \h\, we have 

I ln(a + h) - ln(a) - 1 • h\ , 

hm J— ?: ;/ a ^ = 2.12 

(h,*)-^) l(M)| 

□ 

Definition 2.1.4. TTie Jacobian matrix o/ / a£ a the m x n matrix of 
Df(a) : M n — > M m wi/i respect to the usual bases ofM. n and M. m , and denoted 

/'(«)■ 

Example 2. Let g be the same as in Example 1, then 

g'(a,b) = (-,0) (2.13) 
a 

Definition 2.1.5. A function f : R n -> M m is differentiable onAcR" i/ 

/ is diffrentiable at a for all a E A. On the other hand, if f : A — > IR m , A C 
R™ ; £/ien / called differentiable if f can be extended to a differentiable 
function on some open set containing A. 

2.2 Properties of Derivatives 

Theorem 2.2.1. 1. Iff : W n -> W m is a constant function, then\/a E R n , 

Df(a) = 0. (2.14) 

2. If f : W 1 — > IR m zs a linear transformation, then Va G M n 

D/(a) = /. (2.15) 
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Proof. The proofs are left to the readers □ 
Theorem 2.2.2. // g : M 2 — > K. defined by g(x,y) = xy, then 

Dg{a,b)(x,y)=bx + ay (2.16) 

In oi/ier words, g'(a,b) = (b,a) 

Proof. Substitute p and Dp into L.H.S. of Equation 2.1, we have 

lim \g(a + h,b + k)- g(a, b) - Dg(a, b)(h, k)\ = ^ |fefc| ^ 
(h,fc)-»o 1(^,^)1 (fc,k)->o | (h, k) | 

max(|/i| 2 , \k\ 2 ) 

< lim /' 1,1 1 ' 

(M)-o V/i 2 + k 2 

(2.18) 

< V/i 2 + k 2 (2.19) 
= (2.20) 

□ 

Theorem 2.2.3. // / : R n -> M m differentiable at a, and g : R m -> M p 

differentiate at f(a), then the composition g o f : R n — > R p is differentiable 
at a, and 

D(gof)(a) = Dg(f(a))oDf(a) (2.21) 
Proof Put b — f(a), X — f'(a), fj, — g'(b), and 

u (h) = f(a + h) - f(a) - \(h) (2.22) 
v (k)=g(b + k)-g(b)- f i(k) (2.23) 

for all /» G f " and G l m . Then we have 

|u(/i)| = e(/i)|/i| (2.24) 
=77(fc)|fc| (2.25) 
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where 



lime(/i) = (2.26) 

h—*0 

lining) = (2.27) 



Given h, we can put k such that k = f(a + h) — f(a). Then we have 

\k\ = \X(h) + u(h)\ < [||A|| + e{h)\\h\ (2.28) 



Thus, 



g o f(a + h) - g o /(a) - //(A(/i)) = </(& + fc) - </(&) - //(A(/i)) (2.29) 

= n(k - \{h)) + v(k) (2.30) 
= ii(u(h)) + v(k) (2.31) 



Thus 

\go f(a + h) - go f(a) - n(\(h))\ 



< M\e(h) + [llAH+e^]^) (2.32) 



which equals according to Equation 2.26 and 2.27. □ 



Exercise 1. (Spivak 2-8) Let f : R — > IR 2 . Prove that f is differentiable at 
a G R i/ and on/?/ i/ J 1 and / 2 are, and iaai m £/ws case 



/'(a) = ( (/ p' (a) ) (2.33) 
* (/ 2 )'(a) 1 



Corollary 2.2.4. // / : W 1 -> M m , inen / differentiable at a e R n if and 
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only if each f l is, and 



({n\a)\ 



\'{a) 



\(f m )'(a)J 

Thus, f'(a) is the m x n matrix whose ith row is (/ l )'(a) 
Corollary 2.2.5. // f,g : R™ ^ R are differ entiable at a, then 

D(f + g)(a) = Df(a) + Dg(a) 

D(fg)(a) = g(a)Df(a) + f(a)Dg(a) 

D{f/g){a) = ^-yp , g{a) + 

Proof. The proofs are left to the readers. 



(2.34) 



(2.35) 
(2.36) 

(2.37) 
□ 



2.3 Partial Derivatives 

Definition 2.3.1. // / : R n -> R and a e R n , then the limit 

D if (a) = Jim f^\-^ + h,...,p-f{a\...^) ( ^ g) 

is called the ith partial derivative of f at a if the limit exists. 

If we denote Dj(Dif)(x) to be D i ^(x) ) then we have the following theorem 
which is stated without proof. (The proof can be found in Problem 3-28 of 
Spivak) 

Theorem 2.3.2. If D it jf and Djif are continuous in an open set containing 
a, then 

Dijf(a) = D jti f(a) (2.39) 



16 



y 



-6 -6 



x 



Partial derivatives are useful in finding the extrema of functions. 

Theorem 2.3.3. Let A C W 1 . If the maximum (or minimum) of f : A — > R 
occurs at a point a in the interior of A and Dif(a) exists, then Dif(a) = 0. 

Proof. The proof is left to the readers. □ 

However the converse of Theorem 2.3.3 may not be true in all cases. 
(Consider f(x,y) = x 2 — y 2 ). 

2.4 Derivatives 

Theorem 2.4.1. If f : W 1 — > R m is differentiable at a, then Djf l (a) exists 
for l<i<m,l<j<n and f'(a) is the m x n matrix (Djf l (a)). 
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